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The research literature on open educati^^has 
reported Various studi.es describing and qualifying the terBx^open** in 
education .and in attitudes of teachers involved in such progcaae*^ To 
date, very fev large scale endeavors to assess student achieveaent in 
open education have been coapleted. Studies vhich have been done have 
not shown the hoped for increased gains over acre traditional 
prograas. This paper revievs the pertinent literature on these 
inforaal educational settings, proposes a acre relevant assessaent . 
aodel for cognitive grovth in such prograas utilizing criterion 
referenced aeasureaent, and proposes a acre adequate systea of 
reporting student achieveaent « (Author) 
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Toward Better Assessment of Student ^ 2 
Achievement In Informal Bducatlonal Settings * 
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In the past we have seen an abundance of Innovative instructional 
models being Implemented lii our natlbn*s schools. Most of these 
models have as one of their basic tenets the notion of Individualized 
Instruction. The rationale underlying these individualized models 
stresses the fact that children differ on such variables as Interests » 
attitudes » Intellectual development 1 environmental background, goals 
and so forth. More traditional Instructional models have' not typl- 
. caliy ta^en Into account these Individual differences and perhaps this, 
is why the schools are providing meaningful learning experiences for 
only a small portion of the children. 

Some of the well-knoxm Individualized models Include: Individ- 
ually Prescribed Instruction (Glaser^ 1968) Program for Learning in 
Accordance with Needs (Flanagan, 1967), Mastery Learning (Carroll, 1963, 
1970); and what is most familiarly known in America as Open Education 
( Feathers tbne, 1968a, 1968b; Rathbone, 1971; and Barth, 1972). 

While an abundance of literature is available on these new 
models, many problems remain. The testing component is particularly 
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poorly handled In these new programs. Hambleton .(1973) states: 

It is perhaps surprising to note... that the amount 
of information currently available on the testing 
• methods and decision procedures for these pro- 
grams is quite limited. It is this component that, 
in principle, facilitates the efficient movement 
of students through the instructional .program [p. 3] . 

In particular, the assessment component in -open (or informal) 
educational settings has been poorly^ defined. Barth (1969) states . 
that ".•.the best way of evaluating the effect of the [open] school 
experience on the child is to observe him over a long period of time; 
the best measure of a child* s work is his work.** Although this is a 
logical approach toward assessment, in actual practice it would be 
difficult to "observe over a long period of time" a classroom of 
thirty children. Bussis and Chittenden (1970) imply that part of 
the' reason for the absence of adequate assessment dn open education 
is the lack of suitable measures on several of the student characteris- 
tics. In defense of better assessment, Walberg and Thomas (1972) 
believe that: 

Before. .. [open education] is expanded from the 
limited number of extant experimental settings in 
^ this country, administrators,' teachers and parents 
quite properly should know if it leads to more 
learning, to higher levels of performance in 
reading. c. [etc. ] [p. 207]. 

Purposes 

A number of researchers jifor example, Bussis and Chittenden, 1970) 
feel that a major reason for this poor assessment ^ in open education has 
to do with the fact that the tests employed in the past have been us'ed 
to order children according to more or less intelligence, more or less, 



readiness, and so on; that is, evaluators have used norm-referenced 
assessment- * 

Clearly required Is* a careful look at the testing and measurement 
needs of such Informal educational models. As background to the study, 
there Is a need to review the characteristics and reported research on 
these new programs. The purposes of this study are threefold: (1) to 
describe such models as open-space schools, open classroom schools, the 
Integrated day approach, etc., helping to put further revle^^ and dis- 
cussion of Informal educational settings Into the proper framework; 
(2) to review the pertinent literature on these open models concentrat- 
ing on cognitive growth and assessment of chMdren; and (3) to consider 
testing and measurement problems in open education, proposing a more 
attractive assessment model to measure and Report cognitive growth 
Utilizing criterion-referenced measurement . 

Descriptions of Selected Informal Educational Settings 

Brunetti, Cohen, Meyer and Molner (1972) define open-space schools 
to be: " 

• • • composed of Instructional areas without interior walls, 
ranging in size from tx^o to over thirty equivalent classrooms. 

• . . Open-space schools • . . [can] consist of large open 
areas that can accommpdate the entire student body and 
teaching staff [p. 86]. 

Brunetti, et al» go on to state that: "Teachers (in open-space schools) 

are no longer organizationally isolated but must cooperatively plan the 

activities of several groups of students. The task of planning becomes 

more complex, not only because of the number of students the team is 
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re'sponslble for, but also because teams group and regroup students 
throughout the day and develop complex scheduling plans." 

Open classroom schools are dls tlngi^lshed from open-space schools 
by their lack of vast amounts of architecturally open spac^. While 
open space JLs present to a litnited degree in open classroom plans, 

r 

schools of this nature do not require the integration of students and 
teachers characterized by open-space schools. Open classroom school- 
rooms are usually self-contained and coordinated by one teacher with 
possibly the assistance of a teacher aid. ''These self-contained rooms 
serve as the home base in which students spend the majority of their 
time during the day. Featherstone (1971) states that open classrooms 
are flexibly arranged. They are divided into learning centers to 
provide for the simultaneous occurrence of several activities. Stu- 
dents are not limited to their seats to work, nor does the teacher 
remain in a fixed teaching area. 

The integrated day or free day concept is best described by 
Weber (1971). She explains this approach by stating: 

In planning for the free (or integrated) day there is no 
separation of activities or skills and no separate scheduling 
of any one activity other than the fixed points . . . designed 
for all children in the school. As a result, one might see 
all aspects of the environment — reading, writing, numbers, 
painting, acting, music — In use at all times [p. 90]. 

From Weber's definition we can see that an integrated day approach may 

be the product of an open curriculum but does not necessarily have to 

be so. A traditional teacher may integrate her curriculum including 

arithmetic, language development, etc., without allowing pupils a 

choice in what will be the integrating factor. 
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British primary schools derive their name from the educational 
structure in England. At the present tirae » schooling is divided into 
primary and secondary schools. Primary schools encompass (a^.though not 
always physically) both infant and junior schools. The usual age range 
of children attending these schools is five through seven for the In- 
fant school and eight through eleven for the juniors. Lady Bridget 
Plowden (Lady Bridget Plowden» et al. » 1967) estimates that only one- 
third of the British primary schools can now be characterized as open. 
Consequently, to refer to open education and British primary schools 
synonymously is an error. 




Research on Cognitive Skills in Informal Education 

^ With regard to student achievement in open education, little sub- 



stantial work has been reported. A few empirical studies have been 

made of the effects of architecturally open schools and experimental 

open classroom school programs on selected school outcomes. 

Brunetti, et al. (1972) report that some studies have attempted 
* ■ • 

to show that student growth in both affective and cognitive areas would 

be greater in open-space schools (Bumham, 1971; Kennedy & Say, 1971; 

Myers, 1971). Brunetti reports: . . no negative effects in either 

affective or cognitive growth have heen shown to be associated with 

open space." 

Pavan's (1973) review on research in the nongraded elementary 
school includes three studies examining the effects of student achieve- 
ment in open-space versus traditional environments' (Spencer, 1970; 
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Jeffreys, 1971; Warner, 1971). In all three cases no significant dif- 
ferences were found In student achievement between contrasting groups. 

Gardner (1950, 1965, 1966) conducted longitudinal studies of the 
achievement of children in British integrated day classrooms. Evans 
(1971) concludes that Gardner's overall findings were favorable for 
the British integrated day classrooms compared to British traditional 
classrooms, although the traditional classrooms were not as carefully 
selected as the experimental, integrated day classrooms. 

Tuckman, Cocliran arl Travers (1973) as part of their research 
on the effects of changing to open classroom schools compared the 
achievement of first through fifth graders in open and traditional 
schools using the California Achievement Test . Their results show 
that "standardized achievement was unaffected by the switch to open 
classroom; it was neither improved nor retarded." 

Assessment of Student Achievement in Informal Educational Settings 

We note from the review of cognitive growth research that typi- 
cally the research has involved the use of standardized achievement 
tests* Results showing little or no significant differences between 
open and traditional classrooms x^ere in the majority. The tests used 
in these studies were norm-referenced in nature and it has often been 
noted that the cogi;nitive goals of open educational programs are not 
completely represented on standardized achievement tests* Also it 
should be noted that open education students are not frequently 
exposed to standardized achievement tests and hence their performance 
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may likely be hampered because of a lack of test sophistication. 

A third argument against the use of norm-referenced tests In in- 
formal educational settings concerns Its Inadequacy as an Individual- 
Ized assessment tool. Open educators see norm-referenced testing as 
counterproductive to the goals of their progratas* Their animosity 
stems not so much from an ^Imoslty ^ t:ests per ^e as from the fact 
that test results tend to turn the educator's attention away from 
individualized resources toward an attempt to categorize children 
(Bussls & Chittenden* 1970). T-Jhile norm-referenced testd are of 
limited value for program assessment » they are even less useful for 
classroom monitoring. One alternative to improve program evaluation 
and classroom monitoring is provided by criterion-referenced testing. - 
The assessment component of cognitive areas in open education could 
profit greatly if the proponents of such programs would look beyond 
inadequate testing strategies and integrate objective-based measure- 
ment in their required skill areas. 

A Proposal for Relevanay^Based Testing 4.7% Informal Education 

It is believed that open educators would display much less "ani- 
mosity*' toward testing and assessment if testing were more related to 
the specific decisions that teachers need to make; that Is » if tests 
were constructed not to differentiate among children but to assess the 
actual state of affairs » to measure whether students have achieved the 
criteria by passing through the "threshold" from non-mastery of certain 
predetermined objectives to mastery of those objectives considered by 
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all to be Important for development Into thinking, intelligent adults. 

What we are proposing is a criterion-referenced approach to the 

situation of assessing achievement in open education programs. 

Criterion-referenced tests Have been defined in a variety of ways in 

the literature. , (See, for example, Glaser & Nitko, 1971; Hambleton 

& Novick, 1973.) A very flexible definition has been proposed by 

Glaser and Nitko: 

A criterion-referenced test is one that is deliberately 
constructed so as to yield measurements that are directly 
interpretable in terms of specified performance standards. 
Performance standards are generally specified by defining 
a class or domain of tasks that should be performed by the 
individual. Representative samples of tasks from the do- 
main are organized into a test. Measurements are taken 
and are used to make a statement about the performance of 
each individual relative to that domain [p. 653]. 

Hambleton, Stetz and Rios (1973) provide a decision-making frame- 
work for , criterion-referenced measurement which would benefit teachers'^ 
utilizing such tests. They state that testing is a decision-making 
process; that is, tests are given for the purpose of aiding in making 
decisions. "Decisions relating to mastery of instructional materials 
are best' done with criterion-referenced tests." Test examinees in 
criterion-referenced testing situations consist of two mutually exclu- 
sive groups. One group is made up of examinees with high enough test 
scores to assume they have mastered the material; the second group is 
made up of examinees, who did not achieve the minimum proficiency stan-" 
dard. The establishment of a cut-off score for determining mastery 
level is arbitrary and is primarily a value judgment. 

This decision-theoretic approach tow^^rd testing is most 
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approprlafce for the concerns facing open education teachers. To deter- 
mine effectiveness of instruction and performance of j^dividuals it is 
not necessary to rely upon fixed quota assessment strategies; most 
decisions made in open educational settings are quota free* 

As outlined previously, criterion-referenced tests can be used 
to serve two purposes in open education. First, they can be used to 
evaluate the effectiveness of instruction. Norm- referenced tests 
given at the end of the school year or to compare instruction with 
some control group are usually Inappropriate for making evaluative 
decisions on the effectiveness of instruction due to the fact that 



they are not designed to cover 
criterion- referenced tests are 



the instructional, objectives. However, . 
quite useful to the curriculum evaluator 
because of thp specificity of ^he test results to the curriculum objec- 
tives. '-^ 

Second, criterion-referenced tests can be used to provide very 
specific information on 'the performance levels of individuals on the 
instructional objectives. This information can be used, for example, 
to determine whether an individual has mastered particular objectives. 

This new and. more relevant approach to testing would provide more^ 
information to parents as well as teachers. Parents would be provided 
with performaiice-bas^d data concerning what their children have accom- 
plished in their open education learning experiences. Teachers would 
be provided with Information necessary for the constant decision-making 
situations encountered in such settings. 

To further clarify the intent of criterion-referenced measurement. 
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It should be noted that It would not be necessary to test all students 
at the same time concerning a particular objective or set of objectives. 
In fact, such a procedure would do much to destroy the essence v^f an 
open approach. Individual students or smali^ groups working ^together 
on similar topics could be tested without interruption of classroom 
routine. Such criterion-referenced assessment questions could be in- 
tegrated into the curriculum and included among the activities cards 
popular in most open education classrooms. Tlae emphasis would be 
placed upon the assurance th!at what the children have covered is 
learned, ^d not the more traditional emphasis of testing with all 
its negative connotations. 

The discussion so far has centered around the notion that our 
proposed use of instructional objectives and criterion-referenced test 
items measuring those objectives would be accepted by those in charge 
of informal educational programs. A point of fact is that such a 
proposal could generate a great deal of controversy with such propo- 
nents. The requirement of defining and stating objectives appears 
antithetical to such' a movement. While a nun^er of researchers (for 
example, Ebel, 1973) believe that it is inappropriate to invariably 
use instructional objectives in assessing achievement , it: is possible 
to achieve a more realistic assessment of children's development "^in 
such areas where hierarchical structure and performance tasks are 
easily definable and desirable (i.e., mathematics and reading^. This 
hypothesis should not be extended to more amorphous areas not relying 
upon a structure of hierarchical development nor to the integral 

■ I 
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affective component of children's learning so prevalent in open 
education programs. . • - ■ 

Development of a Move Relevant Reporting System 

Along with a better assessment of student achievement, a more 
representative and systematic approach to reporting student progress 
is needed. A traditional letter-grade approach to reporting student 
progress in such innovative programs is clearly outdated. Addition- 
ally, ^ince most open education objectives are Individualized,^ 
approa'ch which normalizes a class' scores into so many A's, B's^^ C's, 
etc., is completely out of place. Reporting systems utilizing perfor- 
mance" objectives are basically more representative of student achieve- 
ment, but still do not truly represent the individualized^ essence of 
most instruction in open education settings. Most performance-based 
reporting systiems list a grqjjp of performance objectives that all 
students must master to reach criterion in a subject area. Columns 
are xisually provided to check off and record the date when each objec- 
tive is mastered. This assumes that all objectives in a particular* 
subject are imp_ortant for all students. This^approach sfeenis to lose 
the flavor of a truly open environment. In addition, very few systems 
such as the one just mentioned provide for credit to be given for those 
skills and mastered objectives that are completely unique to an indi- 

vidual learner. While it is believed that there are certain skills. 

/ ■ ■ "■ . ..." 

and objectives that should be mastered by all, most emphasis in open 

education relies upon individual differences in Interests and 
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consequently mastery of unique skills and objectives. 

A second argument favoring a more flexible and representative 
reporting syst.em stems from the moqd pervading much of American educa- 
tion today. The demand for accountability from both parents and admin- 
istrators has had th.e effect of forcing teachers to account for their 
actions in the classroom. A ^porting systein which accurately depicts 
a profile of a child's accomplishments , whether they are required or 
elected 9 will help promote a clearer understanding of what children 
are learning in such informal settings. 

T^at is being proposed is a more relevant reporting- system for 

students in open education programs. This reporting system involves 

two main puig)ose9: (1) to allow for adequate reporting of performance 

in those areas deemed 'important for all to master, and (2) to allow for 

c ' , . - , ■ ■ 

credit to be given for those performances, objectives and tasks that 

are unique to an individual student. This system should accurately 

i 

reflect what a student has learned from among the various alternatives 
available in open education programs.. 

This proposed reporting* system could also be used to insure bal- 
ance in a student's learning. Given that open education's assumptions 

v. 

rely upon a child being the prime planner of his learning experiences, 

. \ . 

\ t • 

it is necessary to monitor such activities. This system of reporting 
could act to balance the activities. With actiyities carefully moni- 
ftoted by this system, a teacher could easlly^detect areas in a stu-' 
dent's program where gaps occur through lack of participation in 
certain required subjects. Therefore the proposed "checks and 



balances"v^reporting system would provide checks to allow accurate 
bookkeeping of tasks accomplished » and balances in the curriculum to 
indure adequate coverage of material in various subject areas. Graph- 
ically* this proposed reporting system could possibly resemble a grid 
• incorporating a system of check-offs for what a particular student 
has mastered. 

Expec:ted Contribution to Education 

To date, most research ptudies dealing with open education have 

/ ■ 

concerned themselves with describing and quantifying the term open 
education, and with teacher attitude and opinion toward such programs. 
What is being proposed is a rigorous study toward a body of knowledge 
concerning the studenl:s in these programs. 

The time has come for something to be done on a large scale to 
evaluate objectively the effect open education has on the cognitive 
achievement of children in schools practicing such innovations* 
Little i\j the way of improvement and laudatory announcement can be 
made until a true assessment of the current state of affairs Is made; 
that is whether it "leads to more learning, to higher levels of per- 
formance in ireading," etc. 

This proposal oh the use of criterion-referenced tests will hope- 
fully poii>t the way in the future toward the use of this new and more 
attractive approach to testing in open education, ^^ile previous 
undocumented attempts to study the question of student achievement in 
open education prograir^s have come up with results "slightly" in favor 



-14- 



of a more traditional approach to education or no significant differ- 
ences at all, It is believed that the wrong kinds of tests were used 
(those which purposely spread students out). If such procedures prove 
to be successful, a major advancement in the field of open education 
will be achieved. 
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